Performance evaluation of malaria microscopists working at rechecking laboratories in Ethiopia

Abstract Background Microscopic diagnosis of Giemsa-stained thick and thin blood films has remained the standard laboratory method for diagnosing malaria. High quality performance of microscopists that examine blood slides in health facilities remains critically important. Materials and methods A cross-sectional study was conducted to assess the performance of 107 malaria microscopists working at 23 malaria rechecking laboratories in Ethiopia. A set of 12 blood film slides was distributed to each microscopist. Data was collected and exported to SPSS version 20 for analysis. Chi-square, sensitivity, specificity, percent agreement, and kappa scores were calculated to assess performance in detecting and identification of Plasmodium species. Results The mean age of the participants was 30 ± 5 yrs and most of them (54; 50.5%) were working at regional reference laboratories. Overall, the sensitivity of participants in detecting and identifying malaria parasite species was 96.8% and 56.7%, respectively. The overall agreement on detection and identification of malaria species was 96.8% (Kappa = 0.9) and 64.8% (Kappa = 0.33), respectively. The least accurately identified malaria parasite species was P. malariae (3/107; 2.8%) followed by P. ovale (35/107; 32.7%). Participants working at hospital laboratories had the highest percentage (72.3 %, Kappa=0.51) of accurate species identification. Study participants that had participated in malaria microscopy and quality assurance trainings were significantly better at quantifying parasite densities (P<0.001). Conclusion The accuracy of parasite identification and quantification differed strongly between participants and expert microscopists. Therefore, regular competency assessment and training for malaria microscopists should be mandatory to assure proper diagnosis and management of malaria in Ethiopia.


Introduction
Malaria is a mosquito-borne infectious disease of humans and other animals caused by parasitic protozoans of the genus Plasmodium [1,2].In humans, malaria is caused by P. falciparum, P. malariae, P. ovale, P. vivax or P. knowlesi [3].Globally, an estimated 3.2 billion people in 97 countries and territories are at risk of being infected with malaria and developing disease, and 1.2 billion are at high risk.According to the latest estimates, 214 million cases of malaria occurred globally in 2015 and resulted in 438,000 deaths.Of these, 90% of all occurred in sub-Saharan Africa followed by the WHO Southeast Asia Region (7%) and the WHO Eastern Mediterranean Region (2%) [4].Every minute a child on the African continent dies due to malaria [5].In Ethiopia, malaria remains the leading cause of morbidity and mortality.Approximately 75% of the country is malarious, and 60% of the Ethiopian population (totalling 96.6 million in 2014) is at risk of malaria [6][7][8].
Correct diagnosis of malaria is vital to determine disease prevalence and incidence in order to evaluate the impact of malaria control interventions [9].Training and competency assessment in malaria microscopy using already trained personnel and validated slides are important for evaluating, improving and maintaining high performance in malaria microscopy, and is also an opportunity to provide continuing education [10,11].Many rechecking laboratories were established in different regions of Ethiopia to recheck the blood film slides examined at microscopic sites.Some of the rechecking laboratories are providing only referral services, so that they are not providing routine services on malaria microscopy.The competency status of the malaria microscopists that are working at rechecking laboratories is not known.Therefore this study aimed to assess the performance of malaria microscopists working at Malaria External Quality Assessment (EQA) Rechecking Laboratories in Ethiopia in terms of parasite detection, parasite species identification, stage identification and parasite quantification using light microscopy.

Study setting and participants
The study was conducted from February to May, 2015.Ethiopia is administratively structured with nine regional states and two city administration [12].A total of 23 malaria rechecking laboratories were included in the study (Figure 1).All microscopists working at the malaria EQA rechecking laboratories were invited to participate in this study.

Slide preparation and validation
Whole blood (3-5 ml) was collected in EDTA tubes from volunteers above 18 years of age.A total of 200 blood film slides and Dried Blood Spot (DBS) were prepared from each donor.Six and two microliters were used to prepare a thick and thin blood film, respectively.Both thick and thin blood films were prepared on a single slide.The thin blood film was fixed with absolute methanol.After drying, both thin and thick films were stained for 30-45 minutes with 3% Giemsa working solution.Blood film for microscopic diagnosis of malaria was made semi-permanent by cover slip using Poly mount (toluene based, Cat.# 08381) and a cover glass.Six malaria microscopy experts (microscopists certified as level 1 by WHO after the examination of WHO AFRO external competency assessment) were involved in the validation process and each of them read 2 blood film slides from each donor.Discordant results among validators on species identification were verified by Polymerase Chain Reaction (PCR).

Administration of blood film slides
After validation, two sets of blood film (BF) slides with different values were given to study participants.The first set comprised a total of 10 BF slides; 2 negative slides, 8 positive slides (3 P. falciparum slides with different densities, 2 BF slides with P. vivax, 1 Mixed species slide of P. falciparum and P. vivax, 1 BF slide with P. malariae, and 1 BF slide with P. ovale).These BF slides were used for assessment of parasites detection, stage and species identification.The second set contained 2 positive slides; 1 P. falciparum BF slide with a parasite density of 1891 parasites/µL (with a range of 1419-2363 parasites/µL) and 1 P. falciparum BF slide with a parasite density of 50,659 parasites/µL (with a range of 37990-63323 parasites/µL) were used for the assessment of quantification performance.Based on WHO recommendation, quantification results that deviated ±25% from the mean calculated by expert readers was considered as correct quantification result.A total of 120 minutes (10 minutes per BF slide) were allocated for the exercise [13].

Questionnaire
A structured and standardised questionnaire, which was used to obtain information about participating facilities and microscopists, was distributed amongst the participants.The questionnaire addressed components like age, sex, educational background, type of health facility, in service training, work experience, routine practice of the professional, and number of slides rechecked per person.The data was collected by those that had good experience in malaria microscopy at Ethiopian public health institute laboratories.

Data management and quality assurance
The quality of the blood film slides was checked by reference readers against WHO standards before distribution to malaria microscopists, and the results were critically assessed for completeness.Pre-testing of each questionnaire type was done with 5% of the study population.Intensive supervision was done during data collection.

Statistical analysis
Data was collected and entered into Microsoft Excel sheets and exported to software SPSS version 20 for analysis.Chi -square (for categorical data), sensitivity, specificity, percent agreement, and kappa scores were calculated to assess laboratory professionals' performance in detecting and identification Plasmodium species.Based on WHO classification, microscopists were considered as: 'In training'when the agreement with the expert reader in detection with species identification of malaria parasite was less than 70%; 'Advanced' -when the agreement was greater than or equal to 70% but less than 80%; 'Reference'when the agreement was greater than or equal to 80% but less 90%; and 'Expert' -when the agreement was greater than or equal to 90% [13].Kappa values were calculated to determine the strength of an agreement.Based on the calculation the strength was classified as: < 0.20 slight agreement, 0.21-0.40fair agreement, 0.41-0.60moderate agreement, 0.61-0.80substantial agreement, or 0.81-0.99almost perfect agreement [14].A percent agreement score >80% is a passing point for external quality assessment based on WHO and national guidelines [13,15].

Ethical clearance
This study was done after obtaining approval from the ethical clearance committee of Addis Ababa University College of Health Sciences, Department of Medical Laboratory Sciences, departmental research and ethics review committee.Information about the study was provided to all malaria microscopists involved in the study and to all concerned bodies at the study sites.All participants were assured about the confidentiality, protection and anonymity of the data.Written informed consent was obtained from voluntary study participants before conducting the study.

Results
From a total of 129 malaria microscopists that were engaged in rechecking blood film slides, 107 (83%) participated in the study.The mean age (± SD) of the participants was 30 ± 5 yrs and 90 (84.1%) were men (Table 1).
Based on WHO classification, 78 (72.9%) of the participants were 'in training' with respect to species identification.Only 11(10.3%)participants quantified both BF slides correctly while 34 (31.8%)used a nonrecommended quantification system (Table 2).From a total of 1070 blood film slides that were distributed for detection, 1036 (96.8%) were correctly reported for the presence or absence of malaria parasites and from a total of 856 positive blood film slides only 486 (56.8%) were reported correctly in terms of species identification.Of 214 BF slides that were distributed for quantification, only 63(29.4%) were quantified correctly.
The overall agreement on detection of malaria parasites was 96.8% (Kappa = 0.9), which is 'perfect agreement'  while on identification of malaria species it was 64.8% (Kappa = 0.33) which is a 'fair agreement' (Table 3).False positive rate (negative BF slides reported as positive) and false negative rate (positive slides reported as negative) were 0.84% (7/836) and 11.5% (27/234), respectively.The performance of participants in species identification showed that 80.1% of BF slides with P. falciparum but only 2.8% of BF slides with P. malariae were identified correctly (Figure 2).A total of 64 (61.5%)BF slides with P malariae were identified wrongly as P. falciparum, 42 (58.3%)BF slides with P. ovale were identified wrongly as P. vivax, and 37 (34.6%) of cases with mixed (P.falciparum and P. vivax) BF slides were reported only partially correctly as P. falciparum.
Out of 107 participants, 97 (90.7%) scored <80% overall, which indicates failure based on WHO and national guidelines [13,15].The number of participants that scored a pass mark (≥80%) was higher among participants with MSc level education (6; 31.6%), or participants with more than 5 years work experience (9; 15%) followed by 2-5 years experience (1 participant; 2.3%), respectively (P=0.75).The number of participants that scored ≥80% was higher among participants that had received malaria microscopy and QA training 10 (12.1%) while all participants that had not received training on malaria microscopy and QA scored below 80% (P=0.074).
Participants working at hospital laboratories had the highest percentage agreement with expert readings (72.3%, Kappa=0.51) while regional reference laboratories had the lowest (61.7%, Kappa=0.46;Table 4).
Quantification performance of malaria microscopists working at sub-regional laboratories was 73.3% which was higher than malaria microscopists performance from other malaria rechecking laboratories (P=0.003).Study participants that participated in malaria microscopy and QA training had a better performance on quantification than those who had not received training (P<0.001)(Table 5).

Discussion
Malaria remains a leading cause of morbidity in Ethiopia [7].Microscopy of Giemsa-stained thick and thin blood films is the gold standard method for diagnosis of malaria.Competency assessment is one of the methods to verify microscopists' competency to perform laboratory tests and produce accurate, reliable and timely results.In the current study an agreement between participants and malaria microscopy expert readers in the detection of malaria parasites was 96.8% (Kappa=0.9)which was relatively higher when compared with the study conducted in Hawassa town, Ethiopia which was 88% (Kappa=0.67)[16].Based on WHO recommendation 78 (72.9%) participants were 'in training' in the current study while only 17 (23.6%)were rated as 'in training" in the study conducted in Hawassa town [16].The reason for this may be lack of providing routine service; which is used to develop diagnostic skills for malaria parasite detection and identification, and/or lack of regular training on malaria microscopy.
Both the sensitivity (96.8%) and specificity (96.7%) of microscopists in this study were higher than in the study conducted in Hawassa town which was 82% and 96.5% [16], in USA it was 92% and 90% [17], and in a study from Zambia it was 88% and 91% [18], respectively.The higher sensitivity in the current study regarding the detection of parasites indicates that there was a low rate of false negative results, which means that misdiagnoses of true infections was low.This can help an individual to take appropriate treatment in line with malaria.Overall, strength of agreement in identification of different species of malaria in the current study (Kappa = 0.33) was lower than the study conducted in North Gondar (Kappa = 0.47) [19] and Hawassa town (Kappa=0.63)[16].The number of participants that failed to correctly speciate P. falciparum (19.9%) in the current finding was slightly higher than a similar study conducted in Hawassa, where this was about 18% [20] but lower than a survey conducted in Canada 27% [21].Blood film slides with P.ovale which were identified correctly (32.7%) was higher than the study conducted in North Gondar, northwest Ethiopia in which no microscopist correctly identified P. ovale [22].Low performance on species identification may be due to the fact that some participants did not spend sufficient time to examine the slides, or possibly due to lack of regular training.Lack of correct species identification may lead to incorrect administration of first line treatment.For example the recommended first-line treatment of all clinically and parasitologically-diagnosed uncomplicated P. falciparum malaria in Ethiopia is an artemisinin-based combination therapy (ACT) called artemetherlumefantrine (AL) while the first line treatment for P. vivax is oral chloroquine [6].Correct species identification can be used to treat an individual with an appropriate first line drug and used to prevent drug resistance.
In our study, the false positivity rate (negative BF slides reported as positive) was 0.8% which was lower than the study conducted in Canada (2.0%) [21], 6.9% reported in Hawassa town [16], 7% reported in USA [17], 12% reported in Hawassa, Ethiopia [20], and 19% reported in Democratic Republic of Congo [23].These false positive results could lead to unnecessary treatment with antimalarial drug or a delayed diagnosis of the true cause of illness and misleading the clinician from considering other causes of fever and disease.The false negativity rate (positive slides reported as negative) in our study was 11.5% which was higher than 10.1% reported in Democratic Republic of Congo [23] and 3% reported in USA [17].False negativity can lead to delayed treatment, development of serious complications and possibly death or exposure to unnecessary treatment with other (non antimalaria) drugs.
Quantification performance of malaria microscopists working at sub-regional laboratories was 73.3%, which was higher than malaria microscopist performance from other malaria rechecking laboratories (P=0.003).This might be because most participants that were working at sub-regional laboratories were trained on malaria microscopy and QA training.Study participants that were trained on malaria microscopy and QA performed significantly better on quantification (P<0.001).Correctly-quantified blood film slides in our study were 31.3%, which was lower than 81% of correct quantification reported from the USA [17].Participants that used a non-recommended quantification (+) system in this study was 31.8%, which was lower than the study conducted in Democratic Republic of Congo in which 68.6% of the participants used this quantification system [23].Parasite quantification using non-recommended system may be due to lack of updated information (lack of training), or lack of awareness of the advantage of the quantitative system over a semi quantitative (+) system, or it might be due to lack of commitment to adhere to specified time for counting parasite with the recommended quantification system.This is important because measuring parasite density can be used to monitor patient response to treatment and to study drug efficacy.

Conclusions
Even though the performance of participants was good in detection of malaria using microscopy their agreement with expert microscopists identification of different malaria species and the quantification of parasite densities were very low.Most participants did not identify P. malariae and P.ovale correctly.The ability to identify true positives was lower amongst participants that worked at regional reference laboratories.Participants that were not trained on malaria microscopy displayed very poor performance in parasite quantification.Quantification performance of microscopists working at health centre or hospital laboratories was very low.Therefore, to fill the identified gaps, all stakeholders at all levels have to work on the implementation of regular competency assessments and implement training policy.Demonstration BF slides used for malaria microscopy training have to comprise P. malariae and P.ovale.Regions which are using or which are going to use health centre and hospital laboratories as rechecking sites have to evaluate the performance of laboratories using all external quality assessment methods and have to conduct competency assessment for malaria microscopists that are working at those rechecking sites.

Competing interests
The authors declare that they have no competing interests.

Figure 2 .
Figure 2. Accuracy of malaria microscopists in determining malaria parasite species (8 different known positive BF slides).

Table 1 .
Characteristic of malaria microscopists working on malaria microscopy at malaria EQA rechecking laboratories in Ethiopia (n=107).

Table 2 .
Overall performance in terms of parasite species identification, stage identification and parasite quantification according to WHO classification (n=107).

Table 3 .
Overall sensitivity, specificity and agreement between participants and expert accuracy in detecting and identifying parasite species.

Table 4 .
Agreement between participants (n=107) and expert reader in identification of malaria species, grouped by type of health facility.

Table 5 .
Relationship between quantification performance of malaria microscopists (n=107) and work experience, educational status, types of health facility and training.